Welcome to the¶

NBIS Neural Nets & Deep Learning workshop¶

Who are we?¶

Claudio Bengt Christophe Per
Claudio Bengt Christophe Per
Marcin Ashfaq Malin
Marcin Ashfaq Malin
Per Claudio Bengt Christophe
Claudio

Bengt

Christophe Per
Marcin Ashfaq Malin
Marcin Ashfaq Malin

Who are you?¶

Who are you?¶

Who are you?¶

Who are you?¶

Who are you?¶

Who are you?¶

Course website¶

[https://uppsala.instructure.com/courses/47910](https://uppsala.instructure.com/courses/47910)

  • Learning outcomes
  • Prerequisties/preparations
  • Discussions
  • Schedule / Modules

Feedback session on Friday¶

We are counting on your feedback!

  • Does the schedule work the way it's laid out?
  • What did you think of the lectures and labs?
  • Too simple/hard?
  • Write down one thing you liked and one you didn't at the end of each session, let us know at the end

Practical issues¶

  • Try to keep your cameras on, but microphone muted
  • Lectures with many breaks
  • Take lots of small breaks also when working with the exercises
  • We will try to stick to the schedule, but it's only preliminary until it's happened
  • If you have any questions during the lecture, feel free to unmute and ask. If you don't want to ask in the Zoom meeting, write the question in the chat or on slack

So, this course is about Artificial Neural networks (Anns)¶

ANN2

ANN2

.... What is that?¶

AI vs ML vs ANN vs DL¶

ai_ml_ann_dl

(taken from https://medium.com/ai-in-plain-english/artificial-intelligence-vs-machine-learning-vs-deep-learning-whats-the-difference-dccce18efe7f)


AI -- Artificial Intelligence¶

  • The science and engineering of creating intelligent machines

ML -- Machine learning¶

  • Subset of AI
  • Computer algorithms that learn/improve from experience

ANN -- Artificial Neural Networks¶

  • Subset of ML
  • Model of human neural networks
  • Networks built up by artificial neurons

DL -- Deep Learning¶

  • Subset of ANN (? -- depend on definition of ANN)
  • ANN with more than one hidden layer

Where does ANN come from?¶

History¶

First wave of ANN (funding)¶

  • 1943 First model of biological neruon (McCulloch & Pitts)
  • 1949 Hebbian Learning rules for ANN, (Donald Hebb)
  • 1951 First ANN (Minsky)
  • 1958 The Perceptron -- linear classification
  • 1969 Limitations of the perceptron (Minsky & Papert)







Neuron

(taken from https://www.sciencedirect.com/topics/neuroscience/perceptron)

Neuron

(taken from https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon)

  • multi-variate input
  • singler neuron, step activation (on/off)
  • decision/classification

History¶

Second wave of ANN (funding)¶

  • (1975 Back propagation algorithm (Werbos))
  • 1986 Back propagation rediscovered and sigmoid activation -- ANNs in vogue again
  • 1985-87 ANN conferences, societies started (IEEE, INNS)
  • Vanishing gradient problem
  • SVM outcompetes ANNs, ANNs survives in protein structure prediction (e.g., CASP)

Neuron

(taken from https://www.kdnuggets.com/2019/01/backpropagation-algorithm-demystified.html)

Neuron

(taken from https://predictioncenter.org)

History¶

Third wave of ANN (funding)¶

  • GPUs and parallelization
  • "Deep Learning" buzzword and performing well in contests
  • 1997 Recursive Neural Nets. LSTMs (Smidhuber & Hochreiter)
  • 2000 ReLU (Hahnloser et al.?)
  • 2011 CNNs
  • 2020 AlphaFold2 souvereign at CASP14
  • ANNs are everywhere!

Neuron

Neuron

(taken from https://towardsdatascience.com/a-comprehensive-guide-to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53)

Applications -- general¶

Written and spoke language recognition mnist

(taken from Mouret, Jean-Baptiste & Doncieux, Stéphane. (2008). Evolutionary Intelligence

  • (Autoencoder) showing recogniytion of written text.
    • Also mention sentences more general langauge grammar and grammatics -- NLPN
  • Self-driving cars, risk estimation

Risk predictions mnist

(taken from https://spectrum.ieee.org/computing/embedded-systems/bringing-big-neural-networks-to-selfdriving-cars-smartphones-and-drones)

Applications -- general¶

Forecasting, weather, business

Neuron

(taken from https://www.gjesm.net/article_23079.html)

  • Forecasting, e.g, weather
  • Image recognintion, traioned on images
    • also generation
    • whcih is a real face and which is generated

Face recognition and generation

mnist

(taken from https://medium.com/syncedreview/gan-2-0-nvidias-hyperrealistic-face-generator-e3439d33ebaf)

Applications -- Biosciences¶

Protein structure prediction Neuron

(taken from https://www.theverge.com/2020/12/1/21754310/deepmind-alphafold-ai-protein-folding-casp-competition)

  • Alphafold ooutcompetes other methods in CASP 2020
  • Figure 3: An example of image enhancement.
    • (A) Original malignant mass case extracted from DDSM,
    • (B) Enhanced image using CLAHE, and
    • (C) Histogram representation of the image.

Bioimaging diagnostics bioimaging

(taken from https://peerj.com/articles/6201/)

Applications -- Biosciences¶

Figure 3: An example of image enhancement. (A) Original malignant mass case extracted from DDSM, (B) Enhanced image using CLAHE, and (C) Histogram representation of the image.

Spatial transcriptomics Neuron (taken from https://www.uu.se/en/news-media/press-releases/press-release/?id=5226&typ=pm⟨=en)

Decrypting spatial transcriptomics heterogeneity (spage2vec) in complex tissues

  • resolution
  • balance between intrinsic regul network and extrinsic subcell processes

End of introduction¶

  • pattern recognition
  • non-linear classification/regression?
    • signature verification
    • cybersecurity
  • image/character recognition
    • handwriting (mnist)
    • bioimages
    • faces
    • 3D object recognition
  • forecasting/prediction
    • business
    • medical diagnosis
    • generate images (faces)
  • dimensional reduction
    • bar-coding
  • sequence recognition
    • speech
    • text
    • NLP
  • data mining
  • time series
  • protein structure prediciton